Comparing Estimates to Guessing 1 Running head: Comparing Estimates to Guessing Comparing the Accuracy of Experimental Estimates to Guessing: A New Perspective on Replication and the “Crisis of Confidence” in Psychology

We develop a general measure of estimation accuracy for fundamental research designs called v. The v measure compares the estimation accuracy of the ubiquitous Ordinary Least Squares estimator, which includes sample means as a special case, to a benchmark estimator which randomizes the direction of treatment effects. For sample and effect sizes common to experimental psychology, v suggests that OLS produces estimates that are insufficiently accurate for the type of hypotheses being tested. We demonstrate how v can be used to determine sample sizes to obtain minimum acceptable estimation accuracy. Software for calculating v is included as online supplemental material (R core team, 2012).